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This paper describes the background, method and results of the Arrival Metering 
Precision Study (AMPS) conducted in the Airspace Operations Laboratory at NASA Ames 
Research Center in May 2014. The simulation study measured delivery accuracy, flight 
efficiency, controller workload, and acceptability of time-based metering operations to a 
meter fix at the terminal area boundary for different resolution levels of metering delay 
times displayed to the air traffic controllers and different levels of airspeed information 
made available to the Time-Based Flow Management (TBFM) system computing the delay. 
The results show that the resolution of the delay countdown timer (DCT) on the controllers 
display has a significant impact on the delivery accuracy at the meter fix. Using the “10 
seconds rounded” and “1 minute rounded” DCT resolutions resulted in more accurate 
delivery than “1 minute truncated” and were preferred by the controllers. Using the speeds 
the controllers entered into the fourth line of the data tag to update the delay computation in 
TBFM in high and low altitude sectors increased air traffic control efficiency and reduced 
fuel burn for arriving aircraft during time based metering. 
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I. Introduction 

I N May 2014 the Arrival Metering Precision Study (AMPS) was conducted in the Airspace Operations Laboratory 
at NASA Ames Research Center 1 to provide supporting data for the introduction of technologies developed under 
NASA’s Air Traffic Management Technology Demonstration 1 (ATD-1). ATD-1 introduces NextGen technologies 
for the Terminal area, but also requires improved metering precision in the en route arrival airspace. This paper 
describes purpose, design, method and the results of AMPS. To provide context, we will first briefly introduce 
ATD-1 and today’s en route arrival metering operations. 




Flight Deck Interval Management 
for Arrival Operations 


A. NASA’s Air Traffic Management Technology Demonstration-1 (ATD-1) 

ATD-1 integrates NASA technologies for Flight Deck Interval Management (FIM), Controller-Managed 
Spacing (CMS) tools in the terminal airspace and the Traffic Management Advisor with Terminal Metering (TMA- 
TM) to provide fuel-efficient performance-based arrival operations with high throughput 2 (see Figure 1). TMA-TM 
creates a de-conflicted efficient schedule 


B. Time-Based Metering (TBM) in Center and 
Terminal areas 

The effectiveness of TBM in the terminal 
environment will be impacted by the accuracy at which 
aircraft will be delivered from Center controllers 
working in Air Route Traffic Control Centers (ARTCCs) 
to terminal controllers operating in Terminal Radar 
Approach Control (TRACON) facilities. Figure 2 shows 
a sample current day track plot indicating how aircraft 
are tunneled through an arrival gate, which represents the 
boundary between the Center and the Terminal airspace. 
Figure 2 also shows a large amount of vectoring in both, 
the Center and the Terminal airspace, indicating little 


FIM 


for all aircraft at the runway and all 
merge points. CMS represents a set of 
controller tools that aid Terminal 


controllers in issuing primarily speed 
instructions to meet the schedule. FIM 
represents an airborne component 
intended to enable speed management by 
the flight crew. ATD-1 technologies are 
currently transitioned from NASA to the 
FAA. The ground-based technologies, 
TMA-TM and CMS are expected to 
build the foundation for Time Based 
Metering (TBM) in the Terminal area 
with implementation targeted for the 
FAA’s Time-Based Flow Management 
(TBFM) work package 3 to be 
operational between 2017 and 2020 3 . 


Figure 2: Focus of AMPS study 
in Center (green) and ATD-1 in TRACON (blue) airspace 


Figure 1: Components of NASA’s ATM Technology Demonstration-1 
(ATD-1) 

Focus of 
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Controller-Managed Spacing 
in Terminal Airspace 
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coordination between the two facilities. 

Within the ATD-1 and FAA concept of time -based metering Center and Terminal controllers are expected to 
work together to implement a more efficient arrival plan, generated by the TMA-TM. The TMA-TM will require 
less delay in the Terminal area than the current day TMA. Center controllers will meter aircraft with increased 
precision such that Terminal controllers only need to issue a few speed changes. This way aircraft can remain on 
their routes all the way down to the runway and fly efficient descent profiles with no low altitude vectoring while 
still meeting their scheduled time of arrival at the runway. In order for this concept to work, Center controllers will 
need to deliver aircraft with a high time precision to the terminal boundary meter fixes, likely around 30 seconds. 
However, many Centers today do not conduct time -based metering or are configured to deliver aircraft with only 1 
to 2 minutes accuracy. Therefore, it is important to determine whether and how higher delivery accuracy can be 
achieved with the current or forthcoming TBFM systems for metering in the en route airspace. This question was the 
subject of the Arrival Metering Precision Study (AMPS) reported in this paper. 


II. Problem: Metering Precision in the Center Airspace 

Arrival metering in the Center airspace is performed using schedules and delay information provided by the en- 
route components of the TBFM system. When activated this system relays each aircrafts scheduled time of arrival 
(STA) and current estimated delay value to the air traffic controllers En Route Automation Modernization (ERAM) 
workstation. The ST As and delays are then presented to the Center controllers in meter lists and as a delay 
countdown timer (DCT) located near the aircraft target symbol on the radar controller’s workstation. The delay can 
be configured with different resolutions, thus impacting the metering precision. Evaluating the impact of this 
resolution was the first part of the problem to be addressed in the study and is detailed in the paragraph below. 

Secondly, the current day TBFM system uses the current aircraft speed, assigned altitudes and routing 
information for its underlying trajectory predictions that drive the delay calculation. It has no knowledge of the 
aircrafts’ assigned speed. The new TBFM/ERAM functionality for Ground-based Interval Management - Spacing 
(GIM-S) will advise speeds to meet times and when accepted use the assigned speeds in its computations, thus 
improving the trajectory predictions. This functionality was not available and could not be used at the time of the 
AMPS study. However, AMPS was interested in determining how knowledge of speed intent may impact the 
metering efficiency and precision. This second part of the problem is described after the delay resolution discussion. 


A. Delay Countdown Timer (DCT) resolution 

The first objective of AMPS was to determine the delivery accuracy for different delay resolution values . 


As indicated above, the DCTs on the 
controller’s workstation can be configured with 
different levels of precision. The currently 
available options are displaying the delay rounded 
to 10s of seconds, rounded to the next minute or 
truncated to minutes. Figure 3 provides an 
overview of these options as implemented in the 
AOL’s Multi Aircraft Control System (MACS) 3 4 
research platform which includes a simulation of 
the ERAM Center controller workstation. Note 
how different delay values will be reflected by the 
various DCT resolutions. 

Table 1 illustrates the effect of the resolution 
on how well the actual delay is known. Note that 
the option of displaying delay to the second is not 
available in TBFM today. 


10s of seconds 


1 minute truncated or rounded 
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Figure 1: Meter List and Delay Countdown timer (MACS 
research emulation of Center Controller display) 


Table 1: Delay value and DCT for different resolutions 


Resolution 

Displayed as 

Range 

To the Second (not available) 

00:46 

0:46 

1 0 seconds rounded 

00:50 

00:45 - 00:54 

1 minute rounded 

01 

00:30-01:29 

1 minute truncated 

00 

-00:59-00:59 
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Table 1 illustrates that for an actual delay of 46 seconds, the “10 seconds rounded” resolution will tell the 
controller that s/he has to absorb between 45 and 54 seconds to meet the metering time. In the “1 minute rounded” 
configuration the controller has to absorb between 00:30 and 1:29 minutes to meet the time precisely, but s/he also 
knows that if the indication is “00”, the aircraft’s ETA is within 30 seconds of its STA. Lastly, in the “1 minute 
truncated” configuration, the controller has no further action to take to meet the time, since his or her indication 
shows the aircraft to arrive on time. Therefore, if the requirement for Terminal metering were to deliver aircraft to 
the Terminal boundary within 30 seconds of their STA, a “1 minute truncated” configuration would not allow a 
controller to reliably achieve this precision. By many estimates about 50% of the time -based metering operations in 
the US today are conducted using the “1 minute truncated” configuration. 


determine the impact of using (the controller 


B. Speed Usage by the TBFM Automation 

The second objective of AMPS was to 
entered 4th line) speed intent in TBFM calculations . 

The second issue that was considered to have a potential impact on arrival metering operations in the Center 
airspace was whether or not the TBFM automation used the controller-entered speeds for its predictions. In the 
currently fielded system, TBFM estimates the aircrafts indicated air speed based on its tracked ground speed and the 
wind forecast in that area. This information and nominally adapted descent speeds is then used for the TBFM speed 
predictions through the Center arrival airspace. As a consequence, any speed changes that the controllers instruct the 
flight crews to do will only be reflected over time in the delay values shown in the DCT. Therefore controllers have 
to keep monitoring the DCTs and issue additional instructions if the assigned speed does not have the desired 
impact. Prior research 5, 6 has indicated that not supplying this speed intent to the TBFM automation can be a major 
contributor to controller workload and inefficiencies as multiple instructions may be needed to absorb the required 
delay. Therefore, it was postulated that supplying the speed intent might have a positive impact on efficiency and 
controller workload. As stated before, 

GIM-S functionality was not available 
for the study, so it was decided to supply 
the speed intent to TBFM using a simple 
method that was used successfully in 
prior simulations: Use the speed the 
controller enters in the fourth line, 
interpret it as either Cruise speed, Cruise 
and Descent speed or Descent speed only 
(based on altitude and distance to meter 
fix) and send the cruise/descend speed to 
TBFM. Figure 4 illustrates the effect of 
this method during metering operations: 

The example aircraft DAL671 initially 
required 2:00 minutes delay for a Cruise 



Delay Countdown Timer (DCT) 
+02:00 at Mach .77/280 kts 
+00:50 at 250 kts 


Figure 2: Controller assigned speed updates DCT 

Mach of .77 and a descent speed of 280 kts. The controller opens the speed menu, switches it from M to KTS and 
assigns a speed of 250 kts calibrated air speed (CAS). Given the current aircraft state close to the Top of Descent, 
this speed is interpreted as cruise and descent speed and sent to TBFM. TBFM recalculates the trajectory using the 
new speed values and estimates the required delay at 250 kts (CAS) to be 0:50 seconds. This new delay is indicated 
in the DCT after a few seconds and allows the controller to plan for additional actions if more precision is required. 
This method does not require any new or additional actions by the controller and basically only provides rapid 
feedback on his or her actions. It also requires little extra training. 


III. Approach 

The approach to investigating the metering accuracy was to simulate Center arrival metering operations with 
controllers in the loop interacting with TBFM automation and then vary the two primary parameters of interest: 
DCT resolution and supplying/not supplying speed intent to TBFM. The operations were intended to mimic the 
relevant subset of an environment as expected with TMA-TM in place. TMA-TM coordinates all arrivals in 
generating the schedule for the runways and meter fixes, resulting in even flows to the runways and coordinated, but 
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uneven flows over each meter fix. In order to simulate this environment over just one corner post the meter fix 
acceptance rate over that corner post was restricted. 

An important factor to consider in configuring TBFM was the delay magnitude. If there was no delay to be 
absorbed, controllers could simply let the aircraft fly their original profile to have them arrive on time. Small 
amounts of delay (< 2 minutes) can often be absorbed with speed changes, while larger amounts require heading 
vectors and taking aircraft off route. As a consequence, and supported by previous research 5 6 , the delay magnitude is 
a main contributor to task complexity in arrival metering and needs to be considered. Therefore the simulated arrival 
problems had to mimic the typical delay distribution experienced in metering conditions. This could be achieved by 
using scenarios and meter fix acceptance rates that resulted in 0 to 6 minutes of delay. 

In order to save preparation time and cost AMPS partially re-used traffic scenarios and airspace in Atlanta 
Center (ZTL) from an earlier study. The Rocket High and Dalas Low sectors depicted in Figure 5 (left) feed traffic 
over the northwest comer post along the RPTOR 3 arrival (Figure 5, right) into Hartsfield-Jackson Atlanta 
International Airport (ATL). The airspace could be treated as generic arrival airspace to determine the metering 
accuracy. The study was not intended to look at specific problems within the Atlanta area. 




Figure 3: AMPS test environment: Rocket High and Dalas Low test sectors and RPTOR 3 arrival procedure 

IV. Method 

The AMPS human-in- the -loop simulation was conducted during one week in spring 2014 in the Airspace 
Operations Laboratory at NASA Ames Research Center. One day of training was followed by four days of data 
collection. Experiment design, apparatus, participants and metrics are discussed below. 


A. Experiment Design 

AMPS was designed as a 3x2 experiment 
conditions. These were varied between mns. 
In order to balance the desire for 
randomizing conditions with reducing 
training difficulty and training effects it was 
decided to mn the conditions in blocks. Each 
block exercised all three DCT resolution 
conditions at a single speed intent condition. 
Delay magnitude was varied within each run. 
Figure 6 illustrates the general experiment 
design. Each of the 6 conditions was run 3 
times, resulting in 18 mns of 50 minutes. 
The mn schedule is attached in Appendix A. 
In order to increase experimental power 
without increasing training and data collection 
This resulted in 6 data mns for each condition, 


with three DCT resolution conditions varied over two speed intent 

~ 


Delay Countdown 
Timer resolution 

1 -minute truncated 

1 -minute rounded 

10 seconds rounded 

Varied within blocks 


/ 


/ 


r Wtn 


ignore , use 

4 th line speed: 


Delay magnitude 

high: 4-5 minutes (Vectoring) 
moderate: 2-3 minutes 
Low: 0-1 minutes 

Varied within run 


Varied between blocks 


Figure 4: Experiment Design 

time two controller teams were mn in two separate worlds in parallel, 
totaling 36 data mns. 
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B. Laboratory 

The study was conducted in the areas marked in green and yellow in Figure 7, which depicts the Airspace 
Operations Laboratory at NASA Ames Research Center. Rooms H208 and H200 were configured as controller 
rooms for the parallel worlds. The multi aircraft simulation pilots were situated in the H211 area and four airline 
pilots operated four additional desktop flight simulators in the 280 area. 



Figure 5: Airspace Operations Laboratory layout and lab scenes 


C. Technologies 

The simulation used the following 
technologies: 

The Multi Aircraft Control System (MACS) 4 
and its simulation hub, the Aeronautical Data 
Link and Radar Simulator (ADRS), were used as 
the primary simulation system. This simulation 
environment was developed in the AOL and 
represents a comprehensive air traffic research 
platform with advanced capabilities for scenario 
and target generation, large scale flight deck 
simulation, and controller workstations 
emulating Center, TRACON and Oceanic 
systems. For this simulation the MACS ERAM 
Center controller stations were used. Figure 8 
shows a snapshot of the Rocket High MACS 
ERAM display. Arrival aircraft transit the sector 
from North West to South East. Departures and 
Overflights transit the sector in various 
directions. 

Four Aircraft Simulation for Traffic 
Operations Research (ASTOR) 1 flight deck 
stations, developed at NASA Langley, were 
flown by airline pilots to increase procedure and 
phraseology realism and provide additional data 
points. 



Figure 6: MACS-ERAM display - Rocket High sector 
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The ATD-1 Research Traffic Management Advisor (RTMA) as of April 2014 was 
used as TBFM automation. This version was based on RTMA version 3.12. It 
included all Terminal Metering software, but this was turned off and the standard 
Atlanta (ZTL) adaption was used. However, the following modifications that NASA 
had made were used, which were not part of the fielded version at the time: 

• The TBFM system used a modified cruise/descent Speed logic that 

prevented DCT jumps in response to altitude assignments. 

• All aircraft default descent speeds were set to 280 knots. 

• The TBFM system allowed for processing of cruise/descent speeds from 

the controller stations for the speed intent conditions. 

The TBFM system was operated exclusively by experimenters at the beginning 
of each run to create the desired delay distribution and to make sure both worlds had 
very similar schedules. As mentioned earlier the delay distribution was achieved by 
restricting the arrival rate at the North West gate to 26 - 29 aircraft per hour. This 
reflected a schedule created for Terminal Metering and resulted in small delays at the 
beginning of the rush and larger, varying delays later. A typical example timeline is 
shown in Figure 9. The ST As for each aircraft at the meter fix ERLIN are shown on 
the right; ETAs are shown on the left. Delay values are shown next to the ST As. 
Note the uneven STA distribution indicated by the varying gap sizes between 
subsequent aircraft. This is expected at the meter fix if aircraft from all corner posts 
are scheduled along performance based arrival routes with optimal runway 
utilization. In a situation like this controllers cannot approximate meeting ST As with 
putting aircraft a respective number of miles in trail apart. 

D. Uncertainties 

A number of uncertainties were simulated to increase the realism for the 
controllers and make sure the automation systems would not be computing values for 
an unrealistically perfect environment. These uncertainties were considered 
achievable for the 2017 -2020 primary target time frame of Terminal Metering. 

• Flight technical airspeed errors between target airspeed and actual 

airspeed of 1 0 kts 

• Wind errors between forecast winds in the automation systems and 

environment winds of 1 0 kts RMS 

• Performance modelling errors between automation system models and 
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actual aircraft performance of 5% (this means that TBFM’s mean error 
of estimating the top of descent location was 5% of the descent distance, 
e.g. 5 NM for a 100 NM descent distance) 

All aircraft were assumed ADS-B out equipped. Therefore the position and 
geodetic velocities of all aircraft were known more precisely than in a radar environment. 


Figure 7 Timeline with 
typical schedule during 
AMPS scenario 


E. Participants 

The primary participants of the study were four retired air traffic controllers that had retired within less than a 
year from working the radar controller position. One controller was from Albuquerque Center (ZAB), one from 
Denver Center (ZDV), and two from Oakland Center (ZOA). All had very limited time-based metering experience 
and very little prior exposure to the simulation environment. Field observations and discussions with subject matter 
experts had indicated that many Center controllers in the field may not have much TBM experience or training by 
the time terminal metering will be introduced. Therefore, it was deemed appropriate to evaluate delivery accuracy 
with controllers that had little TBM experience and only a very limited amount of TBM training. If the results had 
indicated infeasibility, a follow-up would have been necessary and additional training requirements had to be 
concluded. However, as the results show, this was not the case. 

Two primary participants worked the test sectors Rocket High and Dalas Low (see Figure 5) in each of the two 
parallel worlds and did not change their position throughout the study. In addition to the primary participants, four 
retired controllers worked high altitude and TRACON ghost positions to feed the traffic in and out of the test 
sectors. Eight general aviation and corporate pilots operated the MACS multi aircraft simulation stations and four 
airline pilots operated the AS TOR stations. 
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F. Operational Procedures 

The following operational procedures were briefed to the controllers: (for reference see Figure 5) The Rocket 
High controller should absorb as much of the metering delay as possible before handing the aircraft off to the Dalas 
Low controller. The high altitude controller needed to ensure that the aircraft had been given the RPTOR 3 arrival 
and the “descend via” clearance. Handoff from high to low needed to occur prior to the waypoints CALCO and 
EUENZ or the aircraft had to be stopped at flight level 240. The goal of the low altitude controller was to meet the 
meter time as close to zero as possible prior to handoff to the TRACON. Accounting for the differences in 
resolution, the low needed as a minimum ensure to achieve +/- 30 seconds for the tens of seconds DCT resolution 
and +/- one minute for the minute rounded and minute truncated DCT resolutions. 


G. Metrics 


Metrics included objective and subjective measures. 

The objective measures included: 

• Delivery accuracy 

• Flight path efficiency 

• Fuel efficiency estimates 

• Number of speed assignments 

• Number of heading/path assignments 

The subjective measures included 

• Controller workload 

• Controller preference 

• Acceptability 

All objective measures used data collected by the MACS and TBFM built in data collection systems. The fuel 
efficiency analysis was done post hoc through an innovative trajectory analysis described in Appendix 2. Most 
subjective measures were taken in post-run, post-block and post- simulation electronic questionnaires that the 
controllers completed at their station. 



Delivery accuracy and flight path efficiency were analyzed using three arcs around the meter fix ERLIN as 
shown in Figure 10. These arcs represent 
approximately the following areas: 

The 125 NM arc approximates the area at 
which the Rocket High altitude controller takes 
the ownership over the aircraft and starts to 
control it. At this arc all aircraft STAs were 
“frozen”, which means assigned by TBFM, but 
no controller had actually tried to absorb the 
delay. Therefore, data for this arc represent the 
entry conditions into the test airspace. 

The 65 NM arc approximates the area in 
which control gets transferred from the high 
altitude controller to the Dalas Low altitude 
controller. Therefore, it can be used to analyze 
the effect of the high altitude controller’s actions 
and represents the entry values for the low 
altitude controller. The 5 NM arc represents the 
area at which aircraft typically get handed off to 
the TRACON and the requirements for terminal 
metering must be met. Measurements at this arc 
are used as exit values from the test airspace and 

are therefore the main metrics of interest. Figure 8; Arcs used for primary met rics 
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V. Results 

This section contains the primary results from the AMPS study in compiled form. In some cases more detailed 
results are available in the appendix. 


1 Minute Rounded / No 4th Line 
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A. Delivery accuracy 

Delivery accuracy was measured as delay (STA - ETA) at the three arcs. The delay was calculated by TBFM 
with an accuracy of tens of seconds. 6 histograms were produced for each arc representing the 6 different conditions. 
The histograms for all arcs are available in Appendix C or this section. Each histogram represents all arrivals that 
crossed the respective arc during 3 data collection runs each in the two worlds. The blue bars show the number of 
aircraft which had a delay value within the respective bin and references the left Y axis. The red line shows the 
cumulative percentage of aircraft with delay values up to the respective bin and references the right Y axis. 

Figure 11 shows the histogram for the 125 NM arc in the “1 Minute Rounded/No 4 th line” condition to illustrate 
the entry conditions into the test airspace. This histogram is representative of all 125 NM histograms as aircraft were 
only minimally controlled 
outside the test airspace. Figure 
11 shows that of the 131 aircraft 
crossing the 125 NM arc during 
the 6 data collection runs, 30 % 
of aircraft entering required less 
than two minutes delay, 
meaning speed control should 
have been sufficient to absorb it. 

40 % of arrivals required two to 
three minutes of delay, meaning 
moderate control actions to 
change speed, altitude and/or 
heading are required. 30 % of 
aircraft required 4 minutes or 
more delay, which could only be 
achieved with substantial 
heading vectors. 

Figure 12 shows the 
histogram for the same condition 
at the 65 NM arc. 143 aircraft 
crossed this arc. These are 12 
data points more than at the 125 
NM arc, because some arrival 
aircraft per scenario were 
initialized between the 125 NM 
arc and the 65 NM arc, which 
required no delay. This explains 
the higher bar at the 0 delay bin. 

Additionally, when the aircraft 
were handed off to the low 
altitude controller the high 
altitude controller had already 
eliminated any delays over 3 

minutes and generally reduced Figure 10: delay at 65 NM arc for 1 minute rounded/no 4 th line condition 

the delays to a manageable value for the low altitude controller. 


Figure 9: 
airspace 


delay at 125 NM arc representing entry conditions into test 


1 Minute Rounded / No 4th Line 



Figure 13 shows the histograms for all conditions at the 5 NM arc. The left column shows from top to bottom the 
three DCT conditions combined with the “Yes 4 th Fine” condition in which controller entered speeds were sent to 
TBFM to update the DCT within seconds. The right column shows the same DCT conditions for the current day 
“No 4 th Fine” condition. The “1 Minute Rounded / No 4 th line” chart reflecting the same condition as Figures 1 1 and 
12 is in the middle right position. Similar histogram sets for the 125 NM arc and the 65 NM arc are available in 
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Appendix C. The histograms at the 5 NM arc in Figure 13 are treated as the delivery accuracy to the TRACON and 
therefore the most important data set for assessing the delivery accuracy that the Center controllers achieved to 
precondition the aircraft for fine-tuning the speeds within the terminal area TBM. 
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Figure 11: Histograms for delivery accuracy at the 5 NM arc around the Meter Fix ERLIN 


Figure 13 shows overall good delivery accuracy across conditions, indicating that the controllers were able to 
successfully meter all arrivals. Each histogram represents 146 to 149 aircraft arrivals crossing the 5 NM arc. Note 
that the peaks at 0 delay also include a number of aircraft that did not require any delay and control instructions. 
This includes the average 2 1 aircraft that did not require delay at the 65 NM arc and several more aircraft that were 
initialized in the low altitude sector at the beginning of the scenario 

The results show that there are differences in the metering accuracy between DCT conditions, but no substantial 
differences for a given DCT resolution in the accuracy between the 4 th line conditions. The “10 seconds rounded” 
conditions at the top show the highest number of aircraft in the 0 second bin with a narrow normal distribution 
around it. This represents the highest metering precision. The “1 minute rounded” condition exhibits more clustering 
because the DCT resolution can only inform controllers whether the aircraft are within 30 seconds of the 0 delay 
value. 30 seconds or more were indicated as a non-zero number in the DCT. Therefore, aircraft are more evenly 
distributed within the -20 to +20 bins. (Note that delays are only known with 10s of seconds’ accuracy) The biggest 
difference between conditions can be seen between the “1 minute truncated” and all other conditions. Since the DCT 
will only indicate when aircraft are within one minute of the 0 delay value, delays are fairly evenly distributed 
between -20 and 50 seconds bins. The reasons for having more aircraft arrive early than late are that (a) the version 
of TBFM used in the study never scheduled an aircraft earlier than its ETA and (b) controllers usually took action 
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until the delay value dropped from “01” to “00”, which was the case as soon as the aircraft had less than 60 seconds 
of delay to absorb. 

To allow a more direct comparison between conditions and an assessment which condition meets certain 
delivery accuracy, Figure 14 combines the cumulative % graphs for all conditions into one plot. The “10 seconds 
rounded” and “1 minute rounded” conditions produced very similar delivery accuracy. The “1 minute truncated” 
condition resulted in larger delays at the meter fix. Almost all aircraft were delivered within +/- 60 seconds. 
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Figure 12: delivery accuracy 5 nm from meter fix ERLIN across all conditions 


The dashed lines in Figure 14 are drawn to indicate which percentage of aircraft achieved a +/- 30 seconds accuracy 
at the meter fix for a given condition. The results from the study are shown in Table 2. 


Table 2: Percent of aircraft arriving within +/-30 seconds at the ERLIN meter fix 


Condition 

More than 30 
seconds late 

Less than 30 seconds early 

Within +/- 30 
seconds 

By DCT condition 

lOsec Rd/Yes 4th 

6.8 % 

100.0% 

93.2 % 

95.3 % 

10 sec Rd/No 4th 

1.4% 

98.7 % 

97.3 % 

1 min Rd/ Yes 4th 

6.2 % 

99.3 % 

93.1 % 

91.4% 

lmin Rd / No 4th 

8.9 % 

98.6 % 

89.7 % 

1 Min Tr/ Yes 4th 

6.8 % 

75.0 % 

68.2 % 

71.9% 

1 Min Tr/No 4th 

4.8 % 

80.3 % 

75.5 % 


As the combined DCT value in the last column of table 2 shows, in this study the controllers delivered 95.3 % 
aircraft within +/-30 seconds to 5 NM arc from the meter fix ERLIN in the “10 seconds rounded” DCT conditions. 
In the “1 minute rounded” conditions they delivered 91.4 % within +/- 30 seconds to the same arc and the “1 minute 
truncated” conditions resulted in 71.9 % of aircraft within +/- 30 seconds at the 5 NM arc. 
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The 4 th line 
condition had little and 
inconsistent impact on 
the delivery accuracy at 
the meter fix. It had 
however an impact on 
the delivery accuracy 
from the high altitude to 
the low altitude 
controller. Figure 15 
illustrates this effect. It 
shows the percentage of 
aircraft within +/-30 
seconds at the three 
different arcs starting at 
the 125 NM arc at the 
bottom.. In all “Yes 4 th 
line” conditions more 
aircraft are within +/-30 Figure 13: percent aircraft within +/-30 seconds at the 3 arcs 

seconds at the 65 NM arc than their corresponding “No 4 th line” condition. This indicates that the high altitude 
controllers were more effective absorbing the delay earlier when they could immediately see the impact of their 
speed assignment on the DCT. 

In the next section we will discuss how this more effective delay management with the 4 th line speed translated 
into increased efficiency. 



B. Path Efficiency 

An analysis of the path efficiency for the six conditions was conducted using only those aircraft that transitioned 
all three arcs. These were between 94 and 97 arrivals per condition with a total n of 576 arrivals. Path efficiency was 
measured as extra track NM between the different arcs. Extra track NM is the difference between the actual NM 
flown between the arcs and the direct distance (=60NM) between the arcs. Figure 16 illustrates the result. Overall 
the “10 seconds rounded” condition without 4 th line input to TBFM resulted in the highest amount of extra track NM 
during the 120 NM section from the 125 NM arc to the 5 NM arc. As discussed before, this condition resulted in the 
best precision but added more than 
10 NM flight distance. The “1 
minute rounded/ No 4 th line” 
condition used only slightly fewer 
extra track NM. The “1 -minute 
truncated” conditions added fewer 
extra NM, but did not result in a 
very accurate flow. This general 
trade-off between precision and 
efficiency was mostly 
compensated for in the “Yes 4 th 
line” conditions, which reduced 
the extra track NM for the more 
precise conditions (10s, 1 minute 
rounded) by 30 % and resulted in 
only slightly less efficient flight 
paths than the most efficient, but 
least accurate condition (“1 minute 
truncated, Yes 4 th line”) 

Flight path length is only an 
indication of efficiency used as a 
stand in, because the simulation 



10-sec 10-sec 1-min 1-min 1-min 1-min 

Rd/No Rd/Yes Rd/No Rd/Yes Tr/No Tr/Yes 


Figure 14: Extra track miles flown between the arcs (High = 125 NM arc to 
65 NM arc, Low = 65 NM arc to 5 NM arc) 
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did not have a reliable fuel model. We were however interested in determining whether the different conditions 
actually resulted in fuel savings. The next section will discuss this analysis. 

C. Fuel Burn Analysis 

Fuel burn was estimated post hoc by synthesizing trajectories that closely matched the actually flown trajectories 
and then computing the fuel for the synthesized trajectories. The analysis took advantage of the fuel bum models 
integrated with the Trajectory Synthesizer (TS) 8 of the Center TRACON Automation (CTAS) 9 . The method is 
described in detail in Appendix B. 

Fuel burn was calculated for arrival flights using the portion of trajectory between a track point 140 NM from the 
meter fix ERLIN and another track point 5 nmi from ERLIN. The distance of 140 NM was selected to include the 
effects of all controller-issued clearances upstream. Only the flights that traversed the full length from the 140 nmi 
distance to 5 nmi were selected for analysis. A few additional flights were eliminated from the analysis due to data 
intermption issues. The results contained a total of 444 flights' fuel burn, corresponding to 74 flights for each of the 
six tool conditions. 
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Figure 15: Comparison of synthesized and actual trajectories for a typical flight. 


All synthesized trajectories were closely fitted to the actual trajectories. Comparison of the synthesized trajectory 
to the actual trajectory showed that the maximum cross track error, the maximum altitude error, and the maximum 
time error were less than 0.3 nmi, 500 ft, and 5 seconds, respectively, for most flights. Only one flight had a 
maximum cross track error greater than 1 nmi, and only one flight (a different one) had a time error greater than 10 
seconds. 

Figure 17 shows the comparison for a typical flight in terms of their horizontal paths (left plot), altitude profiles 
(middle), and the time error (right). 


Figure 18 shows the 
breakdown of the total 
fuel burn by tool 
conditions. The first 
observation is that using 
the 4 th line speed 

reduced the total fuel 
burn by 5% to 9%. The 
second observation is 
that a finer delay 

resolution resulted in 
higher fuel burn. 

Without the 4 th line 

speed, the “10 seconds rounded” delay resolution resulted in 9% more fuel burn than the “1 minute truncated” delay 
resolution. The observations are in accordance with the results of flight path efficiency, indicating a strong 
correlation between flight path length and fuel bum. 
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Figure 16: Fuel burn at different tool conditions 
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The question of why using the 4 th line speeds in TBFM results in more efficient flight paths with less fuel burn 
can be answered by analyzing the control instructions. 

D. Air traffic control instructions 

In order to examine the differences in instructions issued to the aircraft the controller’s data entries were 
analyzed for all runs and all conditions. The general patterns align with the patterns observed for the accuracy and 
the path efficiency analysis. Figures 19 and 20 show the speed and heading/route instructions issued for each of the 
conditions. 
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Figure 19: Speed instructions entered by controllers Figure 20: Heading/route instructions 

Figure 19 shows the number of speed instructions that were issued to the flight crews as well as the number of 
Trial-Speed Assignments. These trial assignments indicated in orange and yellow colors represent assignments that 
the controllers put into the 4 th line, saw the new delay in the DCT and decided to issue a different instruction, 
because they did not like the result. They basically used TBFM to trial plan different speed assignments in order to 
find the most effective one. As a result, the number of issued speed assignment is lower for the “Yes 4 th line” 
conditions than for their respective “No 4 th line” conditions. (Note that controllers were instructed that it would be 
better to input a speed assignment in the Yes 4 th line conditions than using normal speed, because an explicit speed 
assignment would keep the automation up to date better.) In addition to reducing the number of speed instructions 
the more accurate automation feedback from forwarding the 4 th line speeds to TBFM also reduced the number of 
heading clearances in all DCT conditions. 


3.5 


The overall impact of using the 4 th line speeds in TBFM is illustrated in Figure 21. This chart represents all speed 
and heading/route instructions to the 860 arrival aircraft that the controllers entered into the 4 th line of the data tag 
averaged per arrival aircraft. This provides an indication of the average number of instructions the controllers issued 
to each aircraft while it transitioned the 120 NM of arrival airspace. 

Without using the 4 th line speed 
in TBFM controllers issued an 
average of 1.93 speed and .84 
heading/route instructions. With the 
fourth line controllers issued 1.45 
speed and .71 heading/route 
instructions. This is an overall 
reduction of 22% in instructions 
controllers issued and flight crews 
implemented, allowing for fewer 
interruptions during the descent. 
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E. Controller Workload 

Controller workload was 
measured in two ways: in real time 
using a MACS built-in workload 
assessment keypad, modelled after 
the AT WIT methodology 10 and after 
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No 4th Line 
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Figure 21: Comparison of control instruction with and without using 
controller assigned 4 th line speeds in TBFM 
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each run using a modified NASA TLX 1 11 . The 
real time ratings were collected by prompting 
controllers every 3 minutes to rate their 
workload on a scale of 1 to 6. The results show 
that workload was generally adequate with a 
slightly higher workload for the high altitude 
controllers than for the low altitude controllers. 

Figure 22 shows the average workload over 
elapsed run time across all runs by sector. The 
pattern mostly reflects the difficulty/complexity 
of the arrival traffic scenario. As indicated 
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Figure 22: Average WAK ratings across all runs by sector 


earlier, the scenarios were designed with smaller delays in the beginning and larger delays later in the scenario. Prior 
research had shown that the scenario difficulty can have a strong impact on controller workload in this 
environment 6 , which is observed in this study as well. 


There were no statistical differences between DCT and 4 th line conditions as figures 23 and 24 show. 



Figure 23: WAK ratings by DCT condition 
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A modified NASATLX was conducted after each run. The TLX items were significantly different from each 
other, but there was no significant interaction or main effect based on the DCT or 4 th line condition. The data is 
shown in Figures 25 and 26. 
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Figure 25: Modified NASA TLX ratings by DCT condition Figure 26: Modified NASA TLX by 4 th line cond. 


F. Controller Preference 

After each three run block, controllers were asked the following question: “At this point in the simulation, which 
DCT resolution do you think overall is most effective for metering operations?” The responses from the 4 
controllers are depicted in Figure 27. Figure 27 shows the blocks in chronological order and the count of controller 
preferences. The notations in the block identify the controller position who had a different preference than the 
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others, which differed between 
blocks; e.g. W1S5, means that the 
controller in world 1 operating 
sector 5 had this particular 
preference. 

After the first block, which was 
run without sending the 4 th line 
speed to TBFM three of the four 
controllers preferred the “1 minute 
rounded” DCT resolution. After the 
second block with the 4 th line 
speeds used in TBFM and more 
familiarity with the system there 
were always at least three of the 
four controllers who preferred the 
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C; Yes 4th 


D: No 4th 


E; No 4th 


F: Yes 4th 
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Figure 27: Controllers assessment of the most effective DCT. 

10 seconds rounded DCT resolution. After the last block all four controllers stated the “10 seconds rounded” DCT 
resolution as their preference. Only one controller never changed his preference. 

Some controllers commented on their DCT resolution choice. These comments are provided in table 3. 

Table 3: Comments on most effective DCT resolution 


Position Block 4 th Line Choice Comments on DCT Resolution Choice 


World 1- 05 (ZTL) Low 

A 

No 4th 

10 Sec Rounded 

World 1- 05 (ZTL) Low 

B 

Yes 4th 

10 Sec Rounded 

World 1-06 (ZTL) High 

B 

Yes 4th 

10 Sec Rounded 

World 2- 06 (ZTL) High 

B 

Yes 4th 

1 Min Rounded 

World 1-06 (ZTL) High 

C 

Yes 4th 

10 Sec Rounded 

World 1-05 (ZTL) Low 

D 

No 4th 

10 Sec Rounded 


most accurate 

the 10 second rounded is by far the most accurate method of metering 

I PREFER THE 1 MIN ROUNDED BUT, THE 1 0 SEC WAS MORE 
EFFECTIVE 

but 10 sec would be good in list 

I STILL PREFER THE 1 MIN ROUNDED 

you car get very close to a precise time at the fix 


At the end of the training and the end of the study the controllers were asked the following two questions: 

“Which Delay Countdown Timer (DCT) Resolution do you prefer?” 

“Based on what you know of current day controllers & operations, what would be the most practical 
combination of DCT resolution and 4th line update to introduce, train, and use? “ 

Table 4 provides those answers: 


Table 4: Preferred and most practical DCT 



End of Training 

End of Study 

Position 

Preferred DCT 

Most Practical DCT 

Preferred DCT 

Most Practical DCT 

World 1 ZTL 05 Low 

10 Seconds Rounded 

1 Minute rounded 

10 Seconds Rounded 

10 Seconds Rounded 

World 1 ZTL 06 High 

1 Minute Truncated 

1 Minute Truncated 

10 Seconds Rounded 

10 Seconds Rounded 

World 2 ZTL 05 Low 

1 Minute Rounded 

1 Minute Rounded 

1 Minute Rounded 

1 Minute Rounded 

World 2 ZTL 06 High 

10 Seconds Rounded 

10 Seconds Rounded 

10 Seconds Rounded 

10 Seconds Rounded 


Even though it was preferred, one controller noted that the “10 seconds rounded” resolution increased the clutter 
on the controllers’ scope and one controller suggested to use 1 minute rounded next to the aircraft’s target symbol 
and 10 seconds in the meter list. Another controller suggested considering using “1 minute rounded” at the high 
altitude and “10 seconds rounded” at the low altitude position. 

The controllers were also asked after each run how having/not having the 4 th line speed update the DCT 
impacted their overall task/traffic management efficiency. Figure 28 depicts those results. 
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Figure 28: Subjective assessment on whether the 4 th line speeds were used in TBFM had an impact on air 
traffic control efficiency 

The controllers reported that having the 4 th line speeds being used in TBFM had a positive effect on their air 
traffic efficiency. This subjective assessment is supported by the objective data presented earlier in this paper that 
demonstrated the need for fewer instructions and efficiency gains from this condition. 

G. Acceptability 

In the end of study questionnaires, controllers were also asked the following acceptability question: “From your 
point of view, how acceptable is each DCT resolution for current day metering operations when the 4th line 
does/does not update the DCT?” The results are depicted in Figure 29. 
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Figure 29: Acceptability of each DCT resolution for metering ops from controller participants 

The controllers rated the “1 minute truncated” without 4 th line update to the DCT as not acceptable. “1 minute 
rounded” and “10 seconds rounded” without 4 th line update were rated slightly over moderately acceptable. With 4 th 
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line update all DCT resolutions were rated between moderately and very acceptable with the highest acceptability 
ratings for the “1 minute rounded” and the “10 seconds rounded” resolutions. 

VI. Conclusion 

The Arrival Metering Precision Study (AMPS) was conducted in the Airspace Operations Laboratory at NASA 
Ames Research Center to gain insight into two questions relevant for the operational introduction of time -based 
metering in the terminal airspace as developed under NASA’s Air Traffic Management Demonstration- 1 (ATD-1) 
project. The first objective of AMPS was to determine the delivery accuracy for different delay resolution values. 
Within the simulated environment and constraints the following was found: 

All DCT resolutions supported delivering more than 90% of the arrival aircraft within +/- 60 seconds to the 
terminal boundary. Only the “1 minute rounded” and “10 seconds rounded” DCT resolutions supported delivering 
more than 90 % of aircraft within +/-30 seconds to the terminal boundaries. The “1 minute truncated” DCT 
resolution supported a +/- 30 second accuracy only for about 70 % of the arrival aircraft. Most controllers preferred 
the “10 second rounded” DCT resolution; all controllers found the “1 minute rounded” and the “10 second rounded” 
resolutions acceptable. The “1 minute truncated” DCT resolution was found to be less than moderately acceptable. 

The second objective was to determine the impact of using (the controller entered 4th line) speed intent in TBFM 
calculations. Within the simulated environment and constraints it was found that having the controller entered 4 th 
line speeds update the DCT had a very positive impact on control and flight efficiency. Without using the 4 th line 
speeds “1 minute rounded” and “10 seconds rounded” DCT resolutions resulted in more air traffic control 
instructions, additional track miles and fuel burn as compared to the “1 minute truncated” DCT resolution. Using 
the 4 th line speeds in TBFM reduced the number of air traffic control instructions by 22%, the additional track miles 
by 15% to 30 % and the overall fuel burn during a 135 NM long arrival segment by 5% to 9%. Controllers reported 
increased efficiency and higher acceptability ratings resulting from using the 4 th line speeds. 

Based upon these results we recommend (1) using the controller entered 4 th line speed to update the delay in 
TBFM for high and low altitude arrival sectors; (2) letting the controllers/facilities choose their preferred DCT 
resolution separately in data tag and meter list among the “10 seconds rounded” and the “1 minute rounded” DCT 
resolutions and (3) eliminating the “1 minute truncated” DCT resolution in particular when a metering accuracy 
better than +/- 60 seconds is desired. 
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Appendix 

A. Run schedule. One day of training, 4 days data collection with 18 runs, 
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B. Fuel Analysis Methodology 

Since MACS did not have an intrinsic fuel burn model, estimates were made after the simulations were 
completed by analyzing the actually flown trajectories. The analysis took advantage of the fuel burn models 
integrated with the Trajectory Synthesizer (TS) 8 of the Center TRACON Automation (CTAS) 9 . 


TS was originally developed to provide aircraft trajectory scheduling support for CTAS. It was later extended to 
model various flight and pilot procedures with greater flexibility and higher fidelity 12 . TS models an aircraft’s 
trajectory by considering the horizontal path and vertical profile in a decoupled way. The horizontal path consists of 
segments of straight lines and arcs, whereas the vertical profile, including altitude and speed, consists of segments 
with distinct transition control settings. TS has a high-fidelity fuel bum model for large commercial jets. TS does not 
have a high-fidelity model for regional jets, but can be configured to use the Base of Aircraft DAta (BADA) 13 
performance model. This mixture of fuel burn model was adopted for the fuel burn analysis. 


The key step to estimate a trajectory’s fuel burn was to use TS to build a noise-smoothed, synthesized trajectory 
that closely fitted the actual trajectory. To do so, the actual trajectory was first analyzed by the 
CmSimTrackComparer 14 , a research tool that can derive intent information from trajectories. The 
CmSimTrackComparer identified change points for the horizontal path and the vertical profile (altitude and speed) 
of the actual trajectory, respectively. These change points broke up the actual trajectory into horizontal and vertical 
segments that were to be fitted by noise-smoothed segments. The horizontal change points allowed the 
CmSimTrackComparer to fit the horizontal path with a sequence of straight lines and arcs. Similarly, the vertical 
change points allowed the CmSimTrackComparer to fit the vertical profile with a sequence of vertical segments. 
There was more than one way of defining the transition control settings in the vertical segments. For this analysis, 
each vertical segment has a fixed-flight-path angle and a fixed acceleration with respect to the air. The flight-path 
angle and acceleration were computed from the beginning and end track points of each segment by 

( 1 ) 


a 


V e 


t e -h 


y = aretanf— — 


( 2 ) 


Here, a and v stand for acceleration and speed with respect to the air, respectively; b and e are indices for the 
beginning and end track points, respectively; t is for the track time, h is for the altitude, s is for the path distance 
defined by the horizontal path (increasing towards the end); and y for the flight-path angle. These vertical segments, 
combined with the horizontal path segments, allowed the CmSimTrackComparer to create a detailed trajectory 
request for TS. TS then computed the fuel bum required for flying this trajectory. 

Several tolerance parameters were adjusted by trial and error for the segment extraction. For the horizontal path: 

• A heading change tolerance was used to determine if a track point was in a turn. This tolerance must be 
large enough to ignore heading noises and small enough to detect real turns. 

• A maximum heading change was used to determine if a track point belonged to an existing turn or a new 
turn. A smaller value results in finer fitting at the cost of more segments and potential over-fitting. For 
example, a sharp turn can be modeled as one turn of, say, 180 degrees. Alternatively, it can be modeled as 
two turns of 90 degrees each, if the maximum heading change is set to 90 degrees. In the latter case, the 
two turns can have difference radii. 

For the vertical profile, the following parameters must be chosen to be large enough to ignore noises and small 
enough to minimize fitting errors: 

• A maximum flight-path angle change was used to determine if a track point belongs to an existing segment 
or a new segment. 

• A maximum acceleration change was used to determine if a track point belongs to an existing segment or a 
new segment. 
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